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ABSTRACT 



This paper places the competencies to he measured hy the OECD’s Programme for the International 
Assessment of Adult Competencies (PIAAC) in the context of the technological developments which are 
reshaping the nature of the workplace and work in the 2E‘ century. The largest technological force 
currently shaping work is the computer. Computers are faster and less expensive than people in performing 
some workplace tasks and much weaker than people in performing other tasks. On the basis of an 
understanding of the kinds of work computers do well, it is possible to describe the work that will remain 
for people in the future, the skills that work requires and the way that computers can assist people in 
performing that work. The paper argues that a technology-rich workplace requires foundational skills 
including numeracy and literacy (both to be tested in PIAAC), advanced problem-solving skills or Expert 
Thinking (similar to the construct of Problem Solving in Technology-Rich Environments to be tested in 
PIAAC) and advanced communication skills or Complex Communication (not being tested in PIAAC). 



RESUME 



Ce document situe les competences qui seront mesurees dans le cadre du programme de P OCDE pour 
revaluation intemationale des competences des adultes (PIAAC) dans le contexte des avancees 
technologiques qui redessinent la nature du travail et le lieu ou il est effectue au siecle. La plus 
grande force technologique qui actuellement fa 9 onne le travail est I’ordinateur. Les ordinateurs sont plus 
rapides et moins onereux que les individus dans certaines taches sur le lieu de travail, mais bien moins 
performants que les personnes dans Taccomplissement d’autres taches. Si Ton considere que Ton peut 
identifier les types de travaux que les ordinateurs remplissent correctement, il est alors possible de decrire 
ceux qui seront de la responsabilite des individus dans le futur ainsi que les competentes que ces travaux 
necessitent, et la fa^on dont les ordinateurs peuvent assister les individus a les accomplir. L’ argument 
contenu dans le document est qu’un lieu de travail hautement technologique necessite des aptitudes 
fondamentales telles que le calcul et la comprehension de textes (qui seront tons deux testes dans le 
PIAAC), des competences avancees a resoudre des problemes ou Expert Thinking (similaires a Taptitude a 
resoudre des problemes dans un environnement hautement technologique qui sera testee dans le PIAAC) et 
des competences avancees en communication ou Complex Communication (non testees dans le PIAAC). 
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HOW TECHNOLOGY CHANGES DEMANDS EOR HUMAN SKILLS 



Frank Levy 

Department of Urban Studies and Planning 
MIT‘ 

1 . It is a characteristic of labour markets that technology can change the nature of work faster than 
people can change their skills. OECD’s Programme for the International Assessment of Adult 
Competencies (PIAAC) is an ambitious first step toward remedying this problem by constructing a clear 
picture of the current distribution of the skills required to work in technology-rich environments. 

2. If completing the PIAAC required a list of tomorrow’s occupations, the project would be 
hopeless. Innovation is central to market economies and it is impossible to imagine many of the new 
occupations that will exist in a decade’s time. But while we don’t know everything about future 
occupations, we do know something about the skills these future occupations will require. 

3. There is no mystery to this. The largest technological force now shaping work is the computer. 
Computers are faster and less expensive than people in performing some workplace tasks and much weaker 
than people in performing other tasks. By characterising the kinds of work computers do well, we can 
begin to describe the work that will remain for people in the future, the skills that work requires and the 
way that computers can assist people in performing that work. 

4. In this paper, I will argue that a technology-rich workplace requires foundational skills including 
numeracy, literacy and reading ability (all to be tested by the PIAAC), an advanced problem-solving skill I 
will call Expert Thinking (similar to Problem Solving in Technology-Rich Environments to be tested in the 
PIAAC) and an advanced communication skill I will call Complex Communication (not being tested in the 
PIAAC). 

1 ) Three Opening Examples 

We can begin to describe what computers do by considering three workplace tasks: 

• In railroad stations, the task of selling tickets to particular destinations, a moderately skilled task, 
is increasingly performed by self-service kiosks rather than by desk agents. 



flevv@mit.edu . This paper is based on ideas that first appeared in David Autor, Frank Levy and Richard 
J. Murnane, “The Skill Content of Recent Technical Change: An Empirical Investigation’’ Quarterly 
Journal of Economics, 118, 4 (November 2003) pp. 1279-1334; and Frank Levy and Richard J. Murnane, 
The New Division of Labor: How Computers are Creating the Next Job Market, Princeton University 
Press, 2005. 
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• In industrial seaports, the task of recording and tracking the movement of sealed cargo containers 
is increasingly done automatically using signals emitted from RFID tags. The task used to he 
done hy clerks who inspected containers visually. 

• In a doctor’s office, the doctor makes a diagnosis using a computer to access a patient’s 
electronic medical record that contains the patient’s medical history including procedures 
performed hy other doctors. The doctor then uses computerised search to look for potential 
treatments for the patient’s condition. In the past, the doctor would have worked from a paper 
medical record that might have omitted other doctors’ procedures and he would have searched for 
potential treatments in his reference hooks, some of which might have been out of date. 

5. Why do we see this particular mix of outcomes? How do we explain the fact that computers 
substituted for human work in issuing tickets and tracking cargo containers while computers complemented 
the doctor’s diagnostic skills? 

6. The answer begins with two ideas: 

• All human work involves the cognitive processing of information. The financial analyst who 
reads numbers in a spreadsheet, the farmer who looks to the sky for signs of rain, the chef who 
tastes a sauce, the carpenter who feels his hammer as it hits a nail - all these men and women are 
processing information to decide what to do next or to update their picture of the world. 

• Computers execute rules. Some of the rules involve arithmetic (6x9 - 54). Other rules involve 
logical conditions (If [AGE > 35] Go to Statement 13). 

• We can think of a properly running computer program as a series of rules that specify an action 
for each contingency. 

7. When these two ideas are combined with common sense, they say that a computer can substitute 
for a human in processing information when two conditions are present: 

• The information to be processed can be represented in a form that is suitable for use by a 
computer. 

• The processing itself can be expressed in a series of rules. 

8. The first condition is the common sense. The information processing rules in the second 
condition rules can be either deductive or inductive. Deductive rules arise from the logical structure of the 
process. For example, in the case of the railroad ticket kiosk, some deductive rules might be: 

- “If the destination is Philadelphia, the ticket price is $35.50.” 

- “Read the person’s credit card number.” 

- “Send the person’s credit card number to the issuing bank to authorise the charge.” 
Information processing based on deductive rules is often described as rules-based logic. 

9. Inductive rules - a more complicated situation - typically refer to equations based on regressions, 
neural nets and other statistical models where the parameters of the model have been estimated on 
“training samples” of historical cases. The equations with their estimated parameters are then used to 
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process new cases. Information processing based on inductive rules is usually described as pattern 
recognition. It is this pattern recognition that permits a radio receiver to identify the signal from the RFID 
tag on the cargo container. Other pattern recognition software is used to recognise words in voice 
recognition software and to recognise potential fraud in credit card purchasing data. 

10. The third example is different. A computer can complement the doctor’s diagnostic skills by 
providing a full patient history and searching for potential treatments that physician might not otherwise 
find. At the same time, a computer cannot substitute for the diagnostic skills per se. 

11. More precisely, the doctor performs two tasks that cannot be easily computerised. The first 
involves eliciting information from the patient. As any doctor will testify, this is not a simple process. It 
involves listening to the patient’s words. It also means reading the patient’s body language - the tone of 
voice, the avoidance of eye contact or the broken off sentence that indicates the patient is holding 
something back. The doctor must be particularly alert for the famous “last minute” of an appointment when 
the patient, on his way out the door, looks over his shoulder and says “By the way, my wife says I should 
tell you about this pain I have in my stomach.” 

12. Many other jobs today involve similarly complicated human interaction. We can call these 
interactions Complex Communication. 

13. The doctor’s other non-computerised task involves constructing a diagnosis from multiple 
sources of information: what the patient has told him, the patient’s medical history, his knowledge of the 
medical literature, his experience with past cases, and so on. This task has not been computerised because 
it cannot be expressed in a set of step-by-step rules.^ Many other jobs today also involve solving problems 
that lack rules-based solutions. We can call this style of problem solving Expert Thinking. 

14. A few additional comments on Complex Communication and Expert Thinking help to explain 
their importance and the challenges they raise for education and training. 

2 ) Complex Communication 

15. A dozen years ago, at the height of the dot.com mania, experts predicted that the internet would 
eliminate millions of jobs in management, teaching and sales - jobs that involved communicating 
information. Networked computers, people assumed, could communicate the information at much lower 
cost than traditional modes of person-to-person communication. 

16. In practice, these jobs are as numerically important as they ever were. The reason why lies in a 
basic cognitive principle: information is inherently ambiguous and we give information meaning by 
imposing a context. Without a shared context, there is no guarantee that the recipient of information will 
interpret it as the author intended. A good example of this principle comes from what may still be the 
shortest correspondence on record. 

17. The correspondence began with a telegraph consisting of a single character - ? -and the reply was 
a telegraph consisting only of ! 



See for example, Marsden S. Blois, "Clinical Judgment and Computers", New England Journal of 
Medicine, 303 (July 24, 1980), pp. 192-197. 

This example is adapted from Tor Norrestradners, The User Illusion: Cutting Consciousness Down to Size, 
Viking Penguins, 1998. The example has appeared in The Guiness Book of Records as the shortest 
correspondence. 
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18. We don’t do many telegraphs these days hut what would you think if you received a letter 
consisting only of If you had a child away at school you might assume the letter was about money. If 
you hadn’t called your mother recently, you might assume the letter was a pointed reminder. Read hy itself 
- taken out of context - the question mark tells you little. 

19. In fact, this particular question mark was written hy Victor Hugo. In 1862, Hugo finished Les 
Miserables. Exhausted, he dropped the manuscript off with the publisher and left for vacation. Though 
Hugo wanted to relax, he also wanted to know how the book was selling and so he telegraphed “?” to his 
publisher. The book was a smash hit and so the publisher could telegraph back “!” 

20. Hugo and his publisher each knew what was on the other’s mind - they shared a context. 

21. This was the point the dot.com forecasts missed: the job of managers, teachers, sales people and 
others is not to convey information per se but to establish a context in order to convey a particular 
interpretation of information. When a salesperson says you look perfect in lime-green pants, you cannot 
know, based on the verbal information alone, whether the salesperson is being honest. The other things the 
salesperson does - reading your body language, quickly correcting misunderstandings, smiling at 
appropriate times - are designed to establish a context in which you assume you are hearing the truth. 

22. In the same way, writing down formulas on a blackboard is a small part of a calculus teacher’s 
job. The teacher must use examples and back-and-forth conversation to create a context in which students 
can understand what the formulas mean. And asking “What seems to be the problem?” is only the start of 
the doctor’s work to discover and diagnose the patient’s symptoms. 

23. Mastering Complex Communication - the ability to establish a common understanding of 
information - is a highly valuable skill particularly in technology-rich environments where information is 
abundant and circumstances can change rapidly. 

3 ) Expert Thinking 

24. When a problem can’t be solved by rules, it is necessary to look for other solution methods - 
what can be called Expert Thinking. Expert Thinking is a collection of specific solution methods that vary 
with the problem at hand. One frequently used method is what cognitive scientists call case-based 
reasoning. The method is illustrated by an example from automobile repair. 

25. A customer brings in a recently purchased Eiat - a new model - with a non-functioning power 
seat. A technician uses a computerised diagnostic tool to search for problems. We noted earlier that any 
software program, including the software in the diagnostic tool, is a set of rules specifying actions for 
various contingencies. But automotive engineers who write the software can only write rules for the 
contingencies they have anticipated: a faulty switch, a break in the wire connecting the switch to the seat 
motor, a faulty seat motor, and so on. 

26. In other words, the diagnostic tool can solve “known” problems but solving “new” problems 
remains something for humans to do. In a new car, the many new electronic components can interact in 
ways engineers have not foreseen. If the seat problem is caused by one of these unanticipated interactions, 
the factory-programmed rules will detect no error and the technician must solve the problem another way. 

27. In case-based reasoning, the mechanic begins with a kind of pattern recognition in which he 
recognises points of similarity between the current problem and other problems he has solved in the past. 
He uses his previous solutions as a starting point for constructing a new solution - for example, looking for 
failure points that he had seen in analogous problems but that the diagnostic tool did not cover. It is likely 
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that the doctor, diagnosing a patient, uses case-hased reasoning in a similar way to start constructing a 
diagnosis, comparing points of similarity between this patient and other patients he had treated in the past. 

28. What stands out about this problem is that there is no straightforward solution path. That is no 
accident. Problems with straightforward solution paths are increasingly performed by computers and 
complex problems like this will comprise an increasing fraction of human work. It is that fact that makes 
Expert Thinking important. 

4) The Educational and Training Implications of Advanced Skills 

29. It is useful to step away from the argument for a moment to see the educational and training 
implications of these advanced skills. Every teacher knows that rules-based skills are relatively easy both 
to teach and to test. The problem, as we have seen, is that skills that can be codified in rules can also be 
performed by a computer. By their nature. Complex Communication and Expert Thinking cannot be 
reduced to rules and so they are relatively difficult to both teach and assess. 

30. With respect to Expert Thinking, begin with the fact that everyone agrees that children need 
“problem-solving skills.” In practice, however, problem-solving skills have often meant focusing only on 
rules-based solutions like the rules of algebra. The rules of algebra are very important but applying 
algebraic rules is just the second step of a two step problem-solving process. The first step - the step 
computers can’t do - involves examining the messy set of facts in a real-world problem to determine which 
set of algebraic rules to apply - the Expert Thinking. 

31. Today, the labour market values a mechanical engineer’s ability to formulate a problem as a 
particular mathematical model. Once the model is formulated, a computer - not the engineer - will apply 
rules to calculate the actual solution. How does the engineer choose the correct mathematical model? As 
with the earlier cases of the auto mechanic and the doctor, she likely relies on analogies with problems she 
has solved in the past. It follows that her education must include numerous real-world problems to give her 
experience on which to draw - a relatively time -intensive process.^ 

32. Similarly, the skill of Complex Communication cannot be learned by simply reading the right 
book. It requires extensive practice and teacher-student interaction. Similarly, because formulating good 
communication is not a rules-based process, assessments are not easily reduced to multiple choice, 
machine-graded tests. 

5 ) Advanced Skills and Foundational Skills 

33. Common sense says that advanced skills like Expert Thinking and Complex Communication 
must be preceded by a strong foundation in literacy, numeracy and reading, all skills tested in the PI A AC. 
Nonetheless, it is useful to review several mechanisms through which computerisation makes foundational 
skills particularly important. 

34. There is, first, today’s rapid pace of change. An example of the relationship between literacy and 
the rate of change occurred some years ago in a plant assembling electronic controls for missiles. The plant 
was located in the U.S. South and much of the assembly was performed by men and women who had 
worked as agricultural labourers. Many were illiterate and so they could not read assembly diagrams but 



Alan Lesgold argues that computer simulations can be useful in teaching subjects like electronics repair 
because the simulations can generate unusual problems faster than the student would encounter them on the 
job. See Alan Lesgold and Martin Nahernow, “Tools to Assist Learning by Doing”, working paper. 
Learning and Research Development Center, University of Pittsburgh, August 29, 2005. 
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they learned to assemble specific components by watching their neighbours. This method of learning broke 
down because engineers were constantly changing components to address performance problems. The 
result was a stream of change orders from the engineering division to the assembly line. The illiterate 
workers were lost because they could not read change orders. Today, as computers accelerate workplace 
change, being able to understand descriptions of new procedures becomes an ever-more frequent task.^ 

35. A second mechanism linking computerisation and foundational skills involves the way computers 
have transformed concrete processes into numerical abstractions. Some years ago, the Ford Motor 
Company experienced this problem in the course of changing from mechanical carburettors to computer- 
controlled fuel injection. Repairing the computerised modules required an ability to read manuals and the 
ability to mentally connect digitised read-outs on diagnostic tools and the (now invisible) processes they 
represent. A number of mechanics, who were quite skilled repairing physical carburettors, could not make 
this transition. 

36. More generally, as computers have lowered the cost of calculation, numerical tools and models 
now permeate many jobs and holding one of those jobs requires becoming a mathematics consumer. A 
clothing store manager uses a quantitative model to forecast dress demand. A truck dispatcher uses a 
mathematical algorithm to design delivery routes. A bakery worker monitors production using digital 
readouts rather than the smell or feel of the bread. Employees of all kinds are expected to use web-based 
tools to help manage their retirement plans. Each of these tasks involves some aspect of numeracy. In most 
cases, a computerised tool does the actual calculation, but using the model without understanding the 
mathematics leaves one vulnerable to potentially serious misjudgements. 

37. Note that the numeracy, literacy and reading skills described in these examples go well beyond 
what might be called “basic skills”. Being able to multiply and divide will always be important, but they 
are not sufficient to deal with the abstraction of a computerised fuel injection module. Similarly, a typical 
definition of “basic reading skills” is not sufficient to absorb the pages or web views of a repair manual 
including searching for the parts of the manual that apply to the case at hand. 

38. The need for foundational skills applies beyond the workplace to personal and civic life. 
Consider the statement that in 2009, the municipal government of Phoenix, Arizona spent a record amount 
on household garbage collection. In fact, annual spending on garbage collection can reach a record level 
for a variety of reasons - a profligate city government, a growing city population, general inflation, 
households generating more garbage. A citizen can resolve this ambiguity only if he understands that the 
statistic has multiple possible meanings and he knows how to use the web-based resources that provide 
information like population growth. This knowledge, however, requires skills well beyond most definitions 
of “basic” reading and mathematics. 

I return to the issue of foundational skills below. 

How Important Are the Advanced Skills? 

39. Complex Communication and Expert Thinking are only two of the skills used in the economy but 
they are growing in importance. To see this, it is useful to classify all labour force tasks into five broad 
categories:^ 



See also, Theodore Schultz, Transforming Traditional Agriculture, Yale University Press, 1964. 
This categorization was initially developed by Autor, Levy and Murnane, op. cit. 
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• Tasks Requiring Expert Thinking: Solving problems for which there are no rule-based 
solutions. Examples include constructing a diagnosis of a patient’s illness, repairing an 
automobile problem that is not addressed by diagnostic tools, and so on. While computers cannot 
substitute for humans in these tasks, computers can complement human skills by making 
information more readily available. 

• Tasks Requiring Complex Communication; Interacting with humans to acquire information, to 
explain it, or to persuade others of its implications for action. Examples include a manager 
motivating the people whose work she supervises, a sales person gauging a customer’s reaction 
to a piece of clothing, a biology teacher explaining how cells divide, an engineer describing why 
a new design for a DVD player is an advance over previous designs. 

• Routine Cognitive Tasks. Mental tasks that are well described by deductive or inductive rules. 
Examples include maintaining expense reports, filing new information provided by insurance 
customers, and evaluating applications for mortgages. Because these tasks can be accomplished 
by following a set of rules, they are prime candidates for computerisation. 

• Routine Manual Tasks. Physical tasks that can be well described using deductive or inductive 
rules. Examples include installing windshields on new vehicles in automobile assembly plants, 
and counting and packaging pills into containers in pharmaceutical firms. Since these tasks can 
be defined in terms of a set of precise, repetitive movements, they are also candidates for 
computerisation. 

• Non-routine Manual Tasks: Physical tasks that cannot be well described as following a set of 
If-Then-Do rules because they require optical recognition and fine muscle control that have 
proven extremely difficult to program computers to carry out. Examples include driving a truck, 
cleaning a building, and setting gems in engagement rings. Computers do not complement human 
effort in carrying out most such tasks. As a result, computerisation should have little effect on the 
percentage of the workforce engaged in these tasks. 

40. Eigure 1 displays, for the U.S. economy, a picture of the evolution of each of these task 
categories relative to its level in 1959. As the figure shows, tasks requiring Expert Thinking and Complex 
Communication have grown substantially over time while routine tasks - particularly routine cognitive 
tasks that are easily computerised - are now declining sharply. 
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Figure 1 : Trends in Routine and Nonroutine Task Input 
in U.S. Occupations: 1960 to 2002 




Expert Thinking 



-• — Com piex Comm unication — ^ — Nonroutine Manuai Tasks 



— Routine Cognitive Tasks Routine Manuai Tasks 



Source'. Autor, Levy and Murnane (2003) updated to 2002 by David Autor. 

41. The task trends in Figure 1 correspond to a “hollowing out” of the joh structure with the largest 
joh losses coming among clerks, assemhly-line workers, low-level accountants, customer service 
representatives - jobs in the lower middle of the earnings distribution that require rules-based processing of 

'J 

information and rules-based repetitive physical motions . By contrast, jobs requiring Expert Thinking and 

g 

Complex Communication - jobs with relatively higher wages - are growing at rapid rates . As computers 
are increasingly absorbed into the labour market, these trends will almost certainly continue. 



Because repetitive physical motions can be expressed in rules, they are candidates for being performed by 
robots. In developed countries, their rules-based nature also makes the tasks candidates for being sent 
offshore to lower wage countries. See Levy and Murnane (2005) for further discussion. 

Actual trends are likely sharper than those shown in Figure 1 .Figure 1 is based on occupational shifts in the 
economy from 1959-99. Due to data limitations, the task content of each occupation is assumed to be 
constant at its 1978 level. This limitation obscures the task shifts that have occurred within occupations - 
for example, the way in which the development of the automatic teller machine has sharply reduced the 
amount of time a bank teller spends on cashing checks and accepting deposits and has increased the time 
spent on more complicated transactions. 
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The PIAAC Frameworks 

42. To this point, we have been discussing the evolution of the demand side of the labour market. A 
corresponding question applies to the labour market’s supply side: How well are national populations 
prepared to meet the current job market evolution? The PIAAC Frameworks are designed to start to answer 
that question by assessing the levels of three Foundational Skills - Numeracy, Fiteracy and Reading - and 
one Advanced Skill, Problem-Solving in Technology-Rich Environments. 

43. We saw earlier that Foundational Skills are themselves are both deep and broad. Consider the 
meaning of literacy. In today’ s society, effective literacy involves the ability to understand the language on 
an insurance contract, to follow web-based instructions on assembling a chair, to assess the accuracy of a 
candidate’s argument and so on. 

44. The PIAAC Fiteracy Framework is designed to capture this variety. In the Fiteracy Assessment, 
individuals will be asked to process information from different media (e.g. printed text, hyper-linked web- 
based text), information in different formats (continuous text, outlines), information serving different 
purposes (argumentative, narrative, instructions), information that includes graphics as well as text, and 
information that arises in different settings including the workplace and the home. The processing 
assignments will include identifying specific information, understanding and integrating relationships 
among different parts of a text, and evaluating and reflecting on a text’s content. It is easy to be put off by 
the assessment’s complexity until one remembers that literacy in today’s society involves all of these 
situations. 

45. The PIAAC Framework for assessing Numeracy involves a similar breadth of tasks. Many daily 
applications of mathematics are obvious: being able to use postal zone codes, estimating the cost of a 
shopping cart of groceries, understanding the meaning of compound interest in a savings account or a loan 
document. Fess obvious are frequent applications of probability and chance. For example, in the fall of 
2009, a number of newspapers and websites described an unpublished Canadian study suggesting that 
receiving the traditional flu vaccine doubled a person’s chances of getting the HlNl virus.^ It required a 
basic knowledge of probability to recognise that the actual chance of acquiring the HlNl virus was never 
mentioned in these articles - only that the chance was doubled. 

46. The PIAAC Framework on Problem-Solving in Technology-Rich Environments is similar in 
many respects to our discussion of Expert Thinking. In both cases, the problems being posed do not admit 
to rules-based solutions. Where the our examples of Expert Thinking described an individual relying on his 
own past experience (case-based reasoning), the PIAAC Framework focuses on problems where the 
individual relies more heavily on the experience and knowledge of others by using digital technologies - 
acquiring and evaluating information from internet, using email to communicate with others, using digital 
tools to process data where appropriate and so on. The Problem-Solving assessment will examine both how 
the individual uses these skills in work and how the individual solves posed problems. 

47. The PIAAC Framework recognises that solving problems in this way involves two kinds of s ki ll: 
a knowledge of how to use digital tools like email and the internet, and a knowledge of how to structure the 
problem: how to set goals, how to measure progress toward those goals, how to practice what is often 



See for example, Patrick White, “Study Prompts Provinces to Rethink Flu Plan”, The Globe and Mail 
available at http://www.theglobeandmail.com/news/technologv/science/studv-prompts-provinces-to- 
rethink-flu-p%3E%201an/article 1 303330/ 

Because the study was under review by a journal, the authors were unwilling to share the specific results 
with the press - only the “headline” of the doubling. 
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called “metacognition” - reflecting on one’s thinking process - in which an individual decides whether to 
continue with a particular approach to a problem or abandon it in favour of a different approach. 

48. Consider the problem of determining the most effective steps to reduce energy consumption in an 
old house. What step should come first? Triple -pane windows? Putting insulation into the walls? Changing 
the furnace? Are there consistent expert answers to these questions? And do the answers vary by the 
climate in which the house is located? An individual’s ability to make progress on this type of problem is 
one good indicator of their ability to succeed in today’ s economy. 

Conclusion 

49. We opened this paper by arguing that technology can change the nature of work faster than 
people can change their skills. The problem is compounded because a nation’s educational system can 
grow out of touch with job market trends. OECD’s PIAAC Erameworks and Assessments are an important 
step toward closing this gap by describing the skills increasingly needed in the labour market and 
estimating those skills’ prevalence in the population. 

50. No single assessment is the final word on this subject and PIAAC should be seen as a work in 
progress. The current version of PIAAC contains no testing for what we have called Complex 
Communication. Similarly, the computer software in the PIAAC Numeracy Assessment precludes testing 
for an individual’s ability to structure a mathematical problem - an important element of numeracy. 

51. We can expect these problems to be remedied over time. In the interim, PIAAC will serve as an 
important early warning system on the ability of a country’ s workers to deal with rapid technical change. 
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